Beat it! – Gesture-based Prominence Annotation as a Window to Individual Prosody Processing Strategies

In recent work [1], we have suggested a novel approach for fine-grained and fast prominence annotation by naı̈ve listeners. Our approach relies on annotators’ “drummed” replications of a perceived utterance, modulating their drumming velocity in accordance with the perceptual prominence of consecutive linguistic units (syllables, words). The drumming velocity is then used as a fine-grained operationalization of prosodic prominence. This intuitive method exploits the established link between prominence and speech-accompanying gesture [2, 3]. Due to its speed and ease, it allows for the rapid annotation of large amounts of data and yields results that are comparable to fine-grained expert annotations of prominence. In the present study, we evaluated our method further by (1) comparing the intra-sentential prosodic variation as measured with traditional annotations and the drumming method. Our results show that “drummed” prominences capture speaking-style related variability similarly to conventional annotation methods. Additionally (2), we examined whether individual listener strategies can be identified with the help of Random Forests. This method allows for estimating the individual impact of established prominence correlates on prominence impressions. Our analyses unveil individual listener strategies for blending and integrating top-down, bottom-up and context cues into impressions of prosodic prominence.